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ABSTRACT 

Particle swarm optimization (PSO) is a computational method that optimizes a problem by iteratively trying to 
improve a candidate solution with regard to a given measure of quality. It solves a problem by having a population of 
candidate solutions, here dubbed particles, and moving these particles around in the search-space according to 
simple mathematical formulae over the particle's position and velocity. Each particle's movement is influenced by its 
local best -known position (pbest), but is also guided toward the best -known positions (gbest) in the search-space, which 
are updated as better positions are found by other particles. This is expected to move the swarm toward the best solutions. 
The particles move in the search space with considering its own velocity and position called as pbest, but pbest has the 
tendency to flow around the local optima. Because of this problem we compare the different Particle swarm optimization 
based algorithm with its principles & application in this paper. Variable Neighbourhood PSO, Adaptive PSO & Niche 
PSO compare to see the performance of the particles in the search space with respect to time. 
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1. INTRODUCTION 

Particle swarm optimization is a stochastic population-based optimization approach, first published by 
Kennedy and Eberhart in 1995. A large body of research has been done to study the performance of PSO, and to 
improve its performance. Much effort has been invested to obtain a better understanding of PSO. 



Graphical representation of the Ibest model 

Figure 1 

These studies concentrated mostly on a better understanding of the basic PSO control parameters, namely 
the acceleration coefficients, inertia weight, velocity clamping, and swarm size. From these empirical studies, it can 
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be concluded that the PSO is sensitive to control parameter choices, specifically the inertia weight, acceleration 
coefficients and velocity clamping. Wrong initialization of these parameters may lead to divergent or cyclic behavior. 



w 


Graphical representation of the gbest model 

Figure 2 

To gain a better, general understanding of the behavior of particle swarms, in-depth theoretical analyses of 
particle movement in search space is necessary. A few theoretical studies of particle movement can be found, which 
concentrate on simplified PSO systems. 



Graphical representation of the von Neumann model 


2. LITERATURE SURVEY 


Figure 3 


In, James Kennedy and Russell Eberhart [17] Particle swarm optimization as developed by the authors comprises 
a very simple concept, and paradigms can be implemented in a few lines of computer code. It requires only primitive 
mathematical operators and is computationally inexpensive in terms of both memory requirements and speed. Early testing 
has found the implementation to be effective with several kinds of problems. This paper discusses the application of the 
algorithm to the training of artificial neural network weights. 

In, Hui Zhan, Jun Zhang [15] An adaptive particle optimization (APSO) that features better search efficiency that 
classical particle swarm optimization (PSO) is presented. More importantly, it can perform a global search over the entire 
search space with faster convergence speed. The APSO consists of two steps, including exploration, exploitation, 
convergence and jumping out in each generation. It enables the automatic control of inertia weight, acceleration 
coefficient, and other algorithm parameters at runtime to improve the search efficiency and convergence speed. 

In, Julio Barrera and Carlos A. Coello-Coello [28], we will review the most representative PSO-based approaches 
that have been proposed to deal with multimodal optimization problems. Such approaches include the simple introduction 
of powerful mutation operators, schemes to maintain diversity that was originally introduced in the genetic algorithms 
literature (e.g., niching), the exploitation of local topologies, the use of species, and clustering, among others. 
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In Pierre Hansen, Nenad Mladenovic [27]they examine a relatively unexplored approach to the design of 
heuristics, change of neighborhood in the search. Using semantically this idea and very little more i.e only a local search 
routine leads to a new metaheuristic, which is widely applicable. We call this approach variable neighborhood search 
(VNS). Contrary to other metaheuristic based on local search method, VNS does not follow a trajectory but explores 
increasingly distant neighborhoods of the current incumbent solution, and jumps from this solution to a new one if and 
only if an improvement has been made. 

3. PSO 

Canonical PSO Model 

The canonical PSO algorithm was inspired from the behavior of bird flocking and fish schooling [1] [2] [3], The 
two main factors that characterize the status of a particle in the search space are its position and velocity. Eq. (1) is used to 
update the velocity and Eq. (2) is used to determine the new position by adding the previous position and the new velocity 

[7]. 

h , q (f +1) = Ih iq (f) + ac(g*(t) - g iq (t))+ac 2 r 2 (g*(f))- g tq (f) ) (1) 

g if; ( f + 1 )= 8i q (t) +h i q ( t + l ) ( 2 ) 

In eq. (1) and eq. (2 ),h lq ,g iq , g*{t) and g*{t) are the velocity of particle i, a position of the particle, previous 
best position and the best location among all particles in the population respectively. Here r t and r 2 are the random 
functions within a range [o.l] and ac t and ac 2 are the two positive constant parameters called acceleration coefficients, 
which reduce the maximum step size of the particle. The desirable results can be obtained by selecting ac { > ac 2 with 
ac { +ac 2 < 4 and I denotes the inertia factor, while progress the algorithm its value gets reduces from 1.0 to 0. 

In order to monitor the particle in search space area, the maximum movement of velocity in one iteration and the 
position of the particle is limited within the range as in eq. (3) and eq. (4) as, 

K = sign[h iq )min \h iq |, /z max ) (3) 

8i, q =^«Gi,Jmin(|g. ? |,gmax) ( 4 ) 

Here, /z max is the maximum velocity and g max is the maximum solution variable. 

Algorithm 1: The Pseudo Code for PSO Algorithm is explained in Algorithm 1 as Follows: 

Algorithm 1: Pseudo Code of PSO Algorithm 

Input: N, /, ac { and ac 2 , r, and r 2 , h ; , g l .. 

Output: gbest 

Begin_ 

Initialize N number of particles. 

For each particle ie N , 

Initialize velocity lt i and position g t randomly 
Evaluate g r by determining f(g ) using eq. (6) 
if fig,) < fipbest ) for each particle i , then 
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_ Algorithm 1: Contd., _ 

pbest j = g, for each particle i 
end if 

if m'm{f (pbestj V i <f(gbest ) then 
gbest = pbest * 
end if 

Update the particle’s position and velocity using eq. (7) and eq. (8) 
end for 
t = t +1 

Until t > Max _ iterations 

Generally, the PSO algorithm [4] [5] [6] has two basic updating models namely gbest model and pbest model. 
According to eq. (1), the global best (gbest) position g* q (t) is the best particle (obtained by means of fitness function) till 

the t' h iteration and the local best solution (pbest) g* q (t) is the best i' h particle in the t ,h iteration and it is given in eq. (11), 

8f = arg min ", (/ (g*(t ~ \)\f(g, (t ))) (5) 

The gbest and pbest along with r, and r 2 control the effect of new particle velocity. Generally, gbest model 
converges quickly than the pbest model, and also it has the tendency to confine in local optima but pbest model has the 
tendency to flow around the local optima. 

Variable Neighbourhood Particle Swarm Optimization Algorithm (VNPSO) 

VNPSO algorithm, which is a recent metaheuristic algorithm [7] inspired from VNS that depends on the growing 
size of the neighborhood to find the better local optima using some strategies. VNS getaway from the problem of current 
local minimum g* by beginning the local searches from the initiating points experienced from the neighborhood of g*, that 
increases its size repeatedly until it founds better local minima than the current one. If the velocity of the particle is 
dropped below the threshold ve, , then the new velocity is set using the eq. (6), 

vqJthWvf+aqr^gfJf -l) -gjf-l))+ac,r 2 (g*(r -l) -g„(f-l)) (6) 



ve lq 

if 

ve- 

1^1 

IV 

ve x = ( 

U d Ve mzx/ 

{ /p 

if 

ve- 
\ lc >\ 

< ve, 


In eq. (7), u d represents uniform random distribution function, ve max is the maximum velocity and ve iq is the 
velocity of i"‘ particle with a dimension q . The value of two parameters ve, and p are correlated with the performance of 
the algorithm. If ve, is large, then the period of oscillation gets reduced and it increases the probability that the particles 
jump over the local minima for the same number of iterations and also it prevents the particle from converging by keeping 
the particles in the fast flying state. These processes are repeated until it satisfies the termination criteria. 

The pseudo code of VNPSO is summarized in given as Algorithm 2 in the paper. 
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Algorithm 2: Pseudo code Of VNPSO Algorithm 


Initialize N number of particles. 

For each particle is N and each dimension q , 

Initialize velocity Ir and position g t randomly. 

Set Nohope — 0 // Flag of iterations without improvement 

If the end criteria is not met. 

Evaluate each particle’s position depending upon their fitness function 

If the global best position g (t ) is improved, then Nohope = 0 , or else, 

Nohope = Nohope + 1 

Set the local best position. 

end if 

If Nohope < 10 , then 

Update the particle’s position and velocity 

Else 

Update the particle’s position and velocity 

end if 

end if 

t = t +1 

Until f > Max _ iteration 


Adaptive Particle Swarm Optimization Algorithm (APSO) 

Accelerating convergence speed and avoiding the local optima have become the two most important and 
appealing goals in PSO research. However, so far, it is seen to be difficult to simultaneously achieve both goals. For 
example, the comprehensive-learning PSO (CLPSO) in [8] focuses on avoiding the local optima but brings in a slower 
convergence as a result. To achieve both goals, adaptive PSO (APSO) is formulated. Adaptive parameter control strategies 
can be developed based on the identified evolutionary state and by making use of existing research results on inertia weight 
[9]—[12] and acceleration coefficients [13]—[16]. It can perform a global search over the entire search space with faster 
convergence speed. The APSO consists of two main steps. First, by evaluating the population distribution and particle 
fitness, areal-time evolutionary state estimation procedure is performed to identify one of the following four defined 
evolutionary states, including exploration, exploitation, convergence, and jumping out in each generation. It enables the 
automatic control of inertia weight, acceleration coefficients, and other algorithmic parameters at runtime to improve the 
search efficiency and convergence speed. Then, an elitist learning strategy is performed when the evolutionary state is 
classified as convergence state. The strategy will act on the globally best particle to jump out of the likely local optima. 

v d = wv d + c-ijand^pBestf — xfj-t c 2 rand d (nBest 1 d — xf) (8) 

x d = x d + v d (9) 

During the evolutionary process, the velocity and position of particle ion dimension d are updated as where co is 
the inertia weight [9], cl and c2 are the acceleration coefficients [2], and randdl and randd2 are two uniformly distributed 
random numbers independently generated within [0, 1 ] for the d th dimension [ 1 ]. 
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Algorithm 3: Pseudo Code of APSO Algorithm 

Procedure Proposed Algorithm 
begin 

Initialization swarms 
Repeat 

adopt swarms number according to free swarms 
for each swarm do 

if a change is detected in the environment then 
evaluate all pbests then regenerate particles randomly in a 
hyper sphere with radius r centered at gbest and update 
pbests 

update gbest 
else 

if swarm is the best swarm 

do local search around gbest in hyper sphere with radius r 
and update only gbest 
end if 

For each particle in swarm do 
update particle position 
update pbest and gbest 
end-for 
end-if 

test for converge 

end-for 

r exc i=fcmdist 

for each pair of swarms m and n, mon do 
test for conflict if there is delete worse swarm 
end-for 

until a maximum number of fitness evaluations is reached 
end. 


Niche Particle Swarm Optimization Algorithm (NPSO) 

The NichePSO algorithm was proposed in [12]. This approach uses the Guaranteed Convergence Particle Swarm 
Optimizer from van den Bergh and Engelbrecht [13] together with the gbest model, aiming to prevent premature 
convergence of the particles. Additionally, the cognition only model is used to encourage the local search of the particles. 
The method used to identify niches in the NichePSO algorithm is based on keeping track of the changes in the fitness value 
of the particles. If a particle does not show any change within a certain number of iterations, a niche is created, containing 
the particle and its closest topological neighbor. The Euclidean distance is used to determine the closeness of the particles 
so that the closest topological neighbor to the particle selected is the particle within the minimum distance to it. A niche 
radius is computed for each sub-swarm. This radius is the maximum distance between the particles in the sub-swarm and is 
described in equation (TO). 

rj = max{IISxj,g- Sxj.ill] (10) 

This radius is used for adding particles to a sub-swarm and to merge two sub-swarms. If a particle has a distance 
less than rj to the initial particle in a sub-swarm, then the particle is absorbed in the sub-swarm. If the distance between the 
initial particles of two sub-swarm is less than the sum of its radius, then the sub-swarms are merged. This condition is 
described in equation (11). 

MSxjl.g- Sxj2,g II <(rjl + rj2) (11) 

If a sub-swarm has converged to an optimum it is possible that its radius rj has a value of zero. In this case, it is 
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difficult to determine the merge of two sub-swarms. A threshold value p is given and if the distance between initial 
particles of sub-swarms with a radius close to zero is less than p, then the sub-swarms are merged. This is expressed in 
equation (12). 

MSxjl.g- Sxj2,g II < p (12) 

The NichePSO algorithm does not require knowing ahead of time the number of optima of the function. Also, it 
does not require setting up a fixed radius for the niches. However, it depends on a minimum threshold to determine when a 
particle does not show changes and a new niche can be created. It also depends on a minimum distance to determine when 
two sub-swarms that are close to converging to an optimum can be merged. 

Algorithm 4: Pseudo Code of NPSO Algorithm 

Initialize 1 main particle swarm; 

Train main swarm particles using the cognition only model; 

Update fitness of each main swarm particle; 

For each sub-swarm do 

Train sub-swarm particles using one iteration of the GCPSO algorithm; 

Update each particle’s fitness; 

Update swarm radius; 

end 

If possible, merge sub-swarms; 

Allow sub-swarms to absorb any particles from the main swarm that moved into it; 

Search the main swarm for any particle that meets the partition criteria; 

If any is found, then create a new sub-swarm with this particle and its closest 
neighbor; 


4. COMPARISON OF PSO WITH OTHER EVOLUTIONARY METHODS 

Angeline in 1998 [21] did an early study to compare the particle swarm approach and evolutionary computation in 
terms of their performance in solving four nonlinear functions, which have been well-studied in the evolutionary 
optimization literature. He concluded that the performance of the two methods was competitive. Particularly, PSO often 
locates the near-optimum significantly faster than by evolutionary optimization but cannot dynamically adjust its velocity 
step size to continue optimization. 

Kennedy and Spears [22] compared the PSO algorithm with three versions of genetic algorithm (GA), without 
mutation; without crossover; and the standard GA which has a crossover, mutation, and selection, in a factorial time-series 
experiment. They found that all algorithms improved over time, but the PSO found the global optimum on every trial, 
under every condition. In short, PSO appears to be robust and shows superiority over all versions of GA in almost every 
cases. 


Hasen et al. [23] examined the effectiveness of PSO in finding the true global optimal solution and made a 
comparison between PSO and GA in terms of their effectiveness and their computational efficiency by implementing 
statistical analysis and formal hypothesis testing. The performance comparison of the GA and PSO was implemented using 
a set of benchmark test problems as well as two problems of space system design optimization, namely, telescope array 
configuration and spacecraft reliability-based design. They showed that the difference in the computational effort between 
PSO and the GA was problem dependent. It appears that PSO outperforms GA by a large differential in computational 
efficiency when used to solve unconstrained nonlinear problems with 22 continuous design variables and with low - 
efficiency differential when applied to constrained nonlinear problems with continuous or discrete design variables. 
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Lee et al. [24] implemented PSO and compared it with GA to find technical trading rules in the stock market. It 
was found that PSO could reach the global optimal value with less iteration and kept equilibrium when compared to GA. 
Moreover, PSO showed the possibility of solving complicated problems without using the crossover, mutation, and other 
manipulations as in GA but using only basic equations. 

Elbeltagi et al. [25] compared five evolutionary algorithms: GAs, Memetic Algorithms (MAs), PSO, and Shuffled 
Frog Leaping algorithm (SFL) in solving two benchmark continuous optimization test problems. The PSO method was 
generally found to perform better than the other algorithms in terms of the success rate and the solution quality while being 
second best in terms of the processing time. 

Allahverdi and Al-anzi[26] conducted extensive computational experiments to compare the three methods: PSO, 
Tabu search, and Earliest Due Date (EDD) along with a random solution in solving an assembly flow shop scheduling 
problem. The computational analysis indicated that the PSO significantly outperformed the others for difficult problems. 

5. APPLICATION 

Antennas Design 

The optimal control and design of phased arrays, broadband antenna design and modeling, reflector antennas, 
design of Yagi-Uda arrays, array failure correction, optimization of a reflectarray antenna, far-field radiation pattern 
reconstruction, antenna modelling, design of planar antennas, conformal antenna array design, design of patch antennas, 
design of a periodic antenna arrays, near-field antenna measurements, optimization of profiled corrugated horn antennas, 
synthesis of antenna arrays, adaptive array antennas, design of implantable antennas. 

Signal Processing 

Pattern recognition of flatness signal, the design of HR filters, 2D HR filters, speech coding, analog filter tuning, 
particle filter optimization, nonlinear adaptive filters, Costas arrays, wavelets, blind detection, blind source separation, 
localization of acoustic sources, distributed odor source localization and so on. 

Networking 

Radar networks, bluetooth networks, auto tuning for universal mobile telecommunication system networks, 
optimal equipment placement in mobile communication, TCP network control, routing, wavelength division multiplexed 
network, peer-to-peer networks, bandwidth and channel allocation, WDM telecommunication networks, wireless networks, 
grouped and delayed broadcasting, bandwidth reservation, transmission network planning, voltage regulation, network 
reconfiguration and expansion, economic dispatch problem, distributed generation, micro grids, congestion management, 
cellular neural networks, design of radial basis function networks, feed forward neural network training, product unit 
networks, neural gas networks, design of recurrent neural networks, wavelet neural networks, neuron controllers, wireless 
sensor network design, estimation of target position in wireless sensor networks, wireless video sensor networks 
optimization. 

Biomedical 

Human tremor analysis for the diagnosis of Parkinson’s disease, inference of gene regulatory networks, human 
movement biomechanics optimization, RNA secondary structure determination, phylogenetic tree reconstruction, cancer 
classification, and survival prediction, DNA motif detection, biomarker selection, protein structure prediction and docking. 
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drug design, radiotherapy planning, analysis of brain magnetoencephalography data, electroencephalogram analysis, 
biometrics and so on. 

Electronics and Electromagnetic 

On-chip inductors, configuration of FPGAs and parallel processor arrays, fuel cells, circuit synthesis, FPGA- 
based temperature control, AC transmission system control, electromagnetic shape design, microwave filters, generic 
electromagnetic design and optimization applications, CMOS RF wideband amplifier design, linear array antenna 
synthesis, conductors, RF IC design and optimization, semiconductor optimization, high-speed CMOS, frequency selective 
surface and absorber design, voltage flicker measurement, shielding, digital circuit design. 

Robotics 

Control of robotic manipulators and arms, motion planning and control, odour source localization, soccer playing, 
robot running, robot vision, collective robotic search, transport robots, unsupervised robotic learning, path planning, 
obstacle avoidance, swarm robotics, unmanned vehicle navigation, environment mapping, voice control of robots, and so 
forth. 

Design and Modelling 

Conceptual design, electro magnetics case, induction heating cooker design, VLSI design, power systems, RF 
circuit synthesis, worst case electronic design, motor design, filter design, antenna design, CMOS wideband amplifier 
design, logic circuits design, transmission lines, mechanical design, library search, inversion of underwater acoustic 
models, modeling MIDI music, customer satisfaction models, thermal process system identification, friction models, model 
selection, ultra wide band channel modeling, identifying ARMAX models, power plants and systems, chaotic time series 
modeling, model order reduction. 

Image and Graphics 

Image segmentation, auto cropping for digital photographs, synthetic aperture radar imaging, locating treatment 
planning landmarks in orthodontic x-ray images, image classification, inversion of ocean color reflectance measurements, 
image fusion, photo time-stamp recognition, traffic stop-sign detection, defect detection, image registration, microwave 
imaging, pixel classification, detection of objects, pedestrian detection and tracking, texture synthesis, scene matching, 
contrast enhancement, 3D recovery with structured beam matrix, character recognition, image noise cancellation. 

Power Generation and Controlling 

Automatic generation control, power transformer protection, power loss minimization, load forecasting, 
STATCOM power system, fault-tolerant control of compensators, hybrid power generation systems, optimal power 
dispatch, power system performance optimization, secondary voltage control, power control and optimization, design of 
power system stabilizers, operational planning for cogeneration systems, control of photovoltaic systems, large-scale 
power plant control, analysis of power quality signals, generation planning and restructuring, optimal strategies for 
electricity production, production costing, operations planning. 

Fuzzy Systems, Clustering, Data Mining 

Design of neurofuzzy networks, fuzzy rule extraction, fuzzy control, membership functions optimization, fuzzy 
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modeling, fuzzy classification, design of hierarchical fuzzy systems, fuzzy queue management, clustering, clustering in 
large spatial databases, document and information clustering, dynamic clustering, cascading classifiers, classification of 
hierarchical biological data, dimensionality reduction, genetic-programming-based classification, fuzzy clustering, 
classification threshold optimization, electrical wader sort classification, data mining, feature selection. 

Optimization 

Electrical motors optimization, optimization of internal combustion engines, optimization of nuclear electric 
propulsion systems, floor planning, travellingsales man problems, n-queens problem, packing and knapsack, minimum 
spanning trees, satisfiability, knights cover problem, layout optimization, path optimization, urban planning, FPGA 
placement, and routing. 

Prediction and Forecasting 

Water quality prediction and classification, prediction of chaotic systems, stream flow forecast, ecological models, 
meteorological predictions, prediction of the flow stress in steel, time series prediction, electric load forecasting, battery 
pack state of charge estimation, predictions of elephant migrations, prediction of surface roughness in end milling, urban 
traffic flow forecasting and so on. 

6. CONCLUSIONS 

In this paper, we have explained the pseudo code of various PSO, like VNPSO, NPSO, APSO. We have 
compared it with PSO and applications of PSO is also explained in details to get the better idea of PSO . 
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